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ABSTRACT 

This report explores the possibility of using the 
discrepancy measures to assess the implementation of educational 
innovations. The discrepancy measure was derived from the decision to 
segment the educational market, and to select or develop marketing 
strategies, on the basis of discrepancies between the potential user 
and the reguirements of a product. Discrepancies occur becaun^ the 
nature of the product is incompatible with the nature of the user. 
The first section of the report consiaers the discrepancy measure in 
the general context of implementation assessment and indicates that 
the measure makes a potential methodological contribution in this 
area. An analysis of existing data, in the second section of the 
report, indicates that, even in Its present form, the measure 
generates relatively rich data on implementation and permits a rough 
classification of different implementation behaviors. A number of 
modifications which could be made in the measure to make it even more 
suitable for the collection of implementation data are discussed in 
the third section of the report. (Author/DN) 
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INTRODUCTION 

The Far West Laboratory for Educational Research and Development has 

recently been conducting work in the application of marketing techniques 

to the problem of bringing about change In education. In a report on the 

first stage of this work, Sikorski and Hutchlns (1974) describe a study 

In which the concept of market segmentation was applied In the educational 

context. For three Innovative products, schools were grouped Into segments 

that would each require a pc\rt1cular marketing approach from a change agent 

wishing to promote Innovation. The present report Is an outgrowth of that 

study; It Is based upon a secondary analysis of the data generated by the 

measure used to segment the market of schools. This measure, called the 

discrepancy measure, was derived from the decision to segment the market, 

and to select or develop marketing strategies, on the basis of discrepancies 

between the potential user and the requirements of a product. As Sikorski 

and Hutchlns state: 

Discrepancies occur because the nature of the product 
(the need It addresses, the Incentives relevant to Its 
adoption, and Its philosophical slant) Is Incompatible 
with the nature of the user (his needs, his perception 
of Incentives, his beliefs). Discrepancies also arise 
between the circumstances for using the product (resources 
required, adoption convenience) and the resources or 
tolerances of the user. 

Examples of particular discrepancies and of the questions Included in the 
discrepancy measure are presented in Appendix A. 
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This report explores a potential application of the discrepancy 
measure to the assessment of implementation. We decided to explore 
this extension of the measure's usefulness when the measure unexpectedly 
yielded data relevant to innovation implementation. It was found that 
the discrepancy information, which was collected for marketing segmentation 
purposes, appeared to Include considerable data relating to the way the 
innovative products were used in the schools where they had been adopted. 
Thus, Che discrepancy measure showed promise as a way of Illuminating the 
conditions of use and the extent of success of the innovation In the school 
setting. In other words. In addition to providing information that divides 
the market into segments with characteristic discrepancies for an innovative 
product (as discussed by Sikorski and Hutchins), the measure can uncover 
information on what happens after adoption—what problems are encountered 
and what adjustments are made in the implementation process. 

This application of the discrepancy measure may represent a contribution 
to the available methodology in implementation assessment, since it provides 
a way of eliciting self-reports in a relatively non-threatening manner. The 
information it can gather has value for several reasons. The assessment of 
innovation implementation is taking on growing importance in such contexts 
as the evaluation of innovations' effects. Another use for implementation 
assessment data is more germane to dissemination and diffusion: by learning 
more about the different ways a product may be Implemented, the marketer puts 
himself in a better position to work with nonadopters. The information he 
gathers on Implementation problems and their solutions will form patterns 
which are likely to be repeated in the experience of future adopters. One 
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section of this report, based on an exploratory analysis of presently 
available data, provides an example of a set of categories that a 
marketer might use to characterize Implementation patterns. Tlie reader 
will see how a marketer armed with this set of categories would be able 
not only to promote the adoption of an innovative product, but also to 
conduct a marketing campaign designed to forestall or overcome eventual 
problems In the implementation phase. 

The discovery that implementation assessment data has been collected 
by a measure which had marketing analysis as its primary purpose generated 
the present exploration of this additional dimension of the discrepancy 
measure. The report is speculative in nature. Although a data analysis 
is reported, It is an analysis which was conducted on an exploratory and 
tentative basis, and the remainder of the chapter Is primarily hypothetical 
in tone. 

This report first considers the discrepancy measure in the general 
context of implementation assessment, examining Its relationship to the 
major variables and major methodological approaches currently being applied 
in the area. Then, as an example of the application of the discrepancy 
measure to Implementation assessment, an exploratory analysis of available 
data is reported. The report concludes with a discussion of modifications 
in the measure which might make It more suitable for purposes of implementation 
assessment without compromising Its value for market analysis. 
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ASSESSMENT OF IMPLEMENTATION IN ZOUCATIGN 

To examine the particular strengths and weaknesses of a ineasure when 
it is applied to a new field, it is necessary to place it in the context 
of existing approaches in that field. Thus, the value and the limitations 
of the discrepancy measure in implementation assessment must be studied ^ 
against a background of the relevant dimensions of implementation assessment— 
what should be measured; and the major methodological approaches--how it is 
now being measured. In this section, we describe prevailing methods of 
assessing educational innovation implementation, and we discuss the metho- 
dological characteristics of the discrepancy measure in relation to these 
approaches. Next, some major variables involved in implementation assessment 
are described, and the extent to which the discrepancy measure could collect 
information on these variables is discussed. 

Implementation Assess ment Methodology 

Methods of data collection for assessment of implementation include 
the examination of documented records, the observation of implementation, 
and self-report techniques such as personal interviews or self-administered 
questionnaires. The discrepancy measure belongs to the third category, self- 
report techniques. Documented records kept by the implementors can provide 
demographic data, accounts of the planned •Implementation strategies, infor- 
mation about the antecedents and the development of such strategies, and 
descriptive assessments of capabilities to implement. This information can 
provide a baseline undGrstanding of the planned development of implementation 
capability and the recorded outcomes in each implementation phase. Observation- 
al techniques can employ trained observers who structure their observations 
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to assess what is taking place along virtually any dimension relevant to 
evaluation of the implementation. 

Most assessments of implementation must depend primarily upon self- 
report techniques, however. Documented records are usually not structured 
for purposes of implementation assessment, and much of the information they 
contain is inappropriate o< incomplete. Observational techniques may pose 
problems because they are time-consuming and therefore expensive. Since it 
is seldom practical to field an extensive observational effort in an assess- 
ment of implementation, in most cases observational data are collected only 
to provide a validity check for other methods of data collection. Self-report 
techniques, however, can be structured to provide data specifically appropriate 
to assessment of implementation, and the data can be collected at reasonable 
cost. 

But while they are the most viable means of collecting data for assessing 
implementation, self- eport techniques also have limitations from a methodo- 
logical point of view. These limitations have been noted in research reports 
concerned with the assessment of implementation in educational settings. For 
example, one study has pointed to the problem of assessing the degree of 
implementation when there is exclusive or nearly exclusive dependence upon 
subjective personal assessments (Gross, et al, 1971), and another has noted 
evidence that in self-reports of implementation there are often discrepancies 
between what teachers say they are doing and what they are actually doing in 
their classrooms (Goodland, et al, 1970). Analysis of data in the present 
study has indicated that conflicting self-reports are often obtained from teachers 
and principals regarding the same implementation activities. 
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These Inaccuracies in self-reports arise because of differing viewpoints 
and the ubiquitous problem of selective perception. 

The discrepancy measure is also a self-repcrt technique, and, as such, 
it is not free of selective perception validity bias. But this measure 
differs In one important way from other self-report techniques applied to 
the assessment of implementation: the discrepancy measure is an Indirect 
self-report. Other kinds of self- reports tend to be characterized by Inaccuracy 
with regard to negative aspects of implementation because of the reluctance 
of informants to reveal problems that could be construed as failures on their 
part. As an indirect self-report, the discrepancy measure focuses the Informants' 
attention on problems or failures of the product or innovation. By tending to 
focus blame externally In this manner, the measure may lessen an informant's 
sensitivity to discussing problems and failures, although any discrepancies 
discussed may actually be as much the fault of the Implementor as of the product 
or innovation. 

The discrepancy measure's indirectness, which allows a somewhat unobtrusive 
assessment of negative aspects of Implementation, is one of the measure's 
unique potential strengths in the context of Implementation assessment. 
Selective perception and the resultant validity balses are minimized If data 
collection regarding implementation problems can be performed in a manner 
that is not threatening to Informants. The discrepancy measure, by focusing 
attention on the product or innovation, fosters a tolerant atmosphere which 
permits the collection of more complete and accurate data regarding negative 
aspects of the implementation process. 

Implementation Assessment Variables 

Basically, the assessment of Implementation can be conceived as consisting 

o 
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of three sets of variables. The first of these sets of variables involves 
the particioants' Motivation to Accept innovation, the second the D evelopment 
of Implementation Capability , and the third the Implementation Outcome s. 
That is, data collected for implementation assessment generally include back- 
ground information about the school implementation setting and implementation 
plans; information about the motivation, knowledge, and behavior of participants 
in the implementation process; and information about the outcomes, or what is 
taking place once the innovation is considered to be implemented. Each of 
these sets is a composite of individual variables, all of which might be 
examined within the scope of a complete evaluation of an implementation process. 

Although Motivation to Accept innovation can be considered a function of 
a number of lower level variables, these consist of only two basic types. 
One type includes variables describing the characteristics of the implementation 
setting which are relatively set and which are generally accepted as situational 
"givens." An example would be student abilities and achievements. The other 
type includes attitude and opinion variables, such as teachers' views on a 
content area, which are more readily subject to change. The discrepancy measure 
as it is presently constituted does not assess very many fixed characteristics 
of the implementation setting; however, there is no reason why a similar measure 
could not do so. The 'existing measure does assess, at least in an indirect 
manner, attitude and opinion variables relating to motivation to accept innovation. 

Among the variables describing relatively fixed characteristics are included 
some demographic characteristics of the implementors which may determine how 
amenable they are to innovation in general or innovation of a particular kind. 
Such variables as knowledge and prior experience related to the innovation 
(in either a general or specific sense) are relevant in this context. The 
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Information discrepancy concerns staff memebers' knowledge of the particular 
innovation. Student abilities and subject area knowledge may profoundly 
influence motivation to accept an innovation, as in the familiar case where 
decision makers seek improved student performance in a particular subject 
like reading. Some other relatively fixed variables are related to types 
of school organization and may be touched upon by the Administrative Require- 
ments discrepancy. Are the classrooms "self-contained" or "open"; is the 
adiiiini strati ve structure rigidly hierarchical; are students "tracked?" The 
answers to questions like these will clearly influence motives to accept 
a particular innovation, and the researcher should note whether present 
organizational arrangements are like the ones that the innovation tends to 
promote, and also whether the implementors favor a change. 

While not all of the fixed characteristics just mentioned are directly 
assessed by the discrepancy measure, the discrepancy approach in general 
represents a highly appropriate way to assess them. Its underlying philosophy 
is that the congruences and discrepancies between an innovation and a school 's 
present state can provide valuable information about innovation adoption. 
The implications for eventual innovation implementation that can be found in 
similar areas of potential discrepancy suggest strongly that the discrepancy 
approach has value in implementation assessment. 

Motivation to Accept variables of the second type (attitudes and opinions) 
include: attitudes towards the general conceptual or content area of which 
the potential innovation is a part, attitudes towards the specific products 
that are the primary means of implementation, and the extent to which the 
proposed innovation is congruent with educational goals already espoused. 
The discrepancy measure in its present form does assess these variables along 
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with attitudes towards educational Innovation In general, attitudes towards 
competing priorities for educational innovation, and the percieved value 
that the local educational agency and the community place on the particular 
innovation being proposed for implementation. Relevant discrepancy labels 
include Developer, Effect, Priority, Programmed, Structured, Subject Matter, 
and Theory. 

The second major set of implementation variables is that which refers 
to the De velopment of Implementation Capability . This set of variables 
includes the process of cultivating organizational and individual skills and 
orientations necessary for the implementation effort to proceed satisfactorily. 
An assessment of capability development should Include information about 
objectives, procedures, roles and resources. The discrepancy measure probably 
provides more direct and complete information regarding this area of implement- 
ation assessment than any other. 

One major variable concerns the way the implementation effort is structur*ed 
In the educational netting and the way Information about the structure is 
communicated to participants. This variable Includes authority structures, 
declsion-making procedures, and the scope and nature of Implementation plans 
and rules. Those portions of the discrepancy measure which assess the degree 
to which the innovation requires changes or reorganization in the purchasing 
or adopting unit—the discrepancies labeled Disruptiveness and Unit of Adoption-- 
provide information relevant to these implementation variables. 

Another Implementation Capability variable is that of staff development. 
The discrepancy measure provides information relevant to this variable in the 
discrepancy called Training, which assesses the degree of acceptability of the 
teacher training required for effective use of an educational innovation. A 
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third Implementation Capability variable concerns peer relationships. The 
relationships between individual implementors and their peers in their own 
and other schools, especially in the interactions they have concerning the 
innovation, may influence the development of support patterns and implementation 
capability. The discrepancy measure touch??s upon this aspect of implementation 
in its assessment of the degree to which an individual teacher perceives others 
as approving the innovation. This assessment comes under the headings of 
Bandwagon and Peer Opinion. 

A final Implementation Capability variable is that of resources. This 
variable includes the extent to which adequate planning time, instructional 
materials and resources, and information about their availability are present 
as part of the implementation effort in an educational setting. The discre- 
pancy measure collects information relevant to this implementation variable 
through its assessments of discrepancies in such areas as Cost, External 
Support, Materials, Personnel, Space, and Time Required. 

The last major set of variables is that of Implementation Outcomes. 
Implementation Outcomes can be considered a function of quantity, quality, 
and durability variables. Although Implementation Outcome variables are, 
of course, closely related to Motivation to Accept and Implementation Capability 
variables, the discrepancy measure probably provides less Information directly 
relevant to Implementation Outcomes than to the other two sets of Implementation 
variable*;.. 

The variable of Implementation quantity might include such facts as the 
number of administrative units involved in the implementation effort, the 
number of participants (both Implementors and students), and the distribution 
of the participants on demographic characteristic variables. The discrepancy 
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measure touches on this variable when It assesses, for example, what porportion 
of a school's teachers have used a Minicourse. Also included in this variable 
would be the number and distribution of innovation activities and products, 
the amount of time devoted to implementation, and the distribution of that 
time. Such discrepancy measures as Materials and Time Required may yield 
some information on implementation quantity. However, as it is presently 
structured the discrepancy measure does not provide extensive data on this 
variable. 

The same may substantially be true with regard to the variable of 
implementation quality, although the discrepancy measure provides opportunities 
for indirect inferences regarding implementation quality. Implementation 
quality includes the relationship between goals and objectives and capabilities, 
and the extent to which goals are met. The quality variable also covers the 
systematic examination and selection of materials and activities furthering 
innovation, the degree of preparation for intensive implementation, the use 
of particular instructional techniques, and exceptional student achievement 
and retention. 

The implementation quality variable could also include consideration 
of the extent to which materials were adapted for special needs, or the innova- 
tion was put into practice beyond the specified implementation procedures. In 
these aspects of implementation quality assessment the discrepancy measure 
might yield relevant information. One portion of the discrepancy measure. 
Adaptability, concerns the degree to which the innovation is adaptable or 
amenable to revision for the specific purposes of the innovator, and this 
information could include evidence of creative adaptation. 



12 



The last major Implementation Outcome variable Is Innovation durability. 
Inferences from other outcome variables, such as measures of large Implementa- 
tion quantity and high Implementation quality, are often used to determine 
or estimate durability. In addition, future plans for Implementing the 
Innovation or following up present Implementation efforts may be examined. 
The discrepancy measure provides little direct Information regarding Implemen- 
tation durability, but It may provide Indirect Information. For example. It 
might be assumed that an Implementation situation In which a great deal of 
discrepancy Is measured will probably not result In Innovation durability. 
This kind of assumption could be tested as a further check on the measure's 
validity. 

In general, the discrepancy approach does appear to be capable of providing 
information relevant to the assessment of implementation. Still, although the 
approach has considerable potential usefulness in this regard, the existing 
discrepancy measure is not uniformly applicable over the entire range of 
implementation variables. It provides the most direct information in the 
area of Development of Implementation Capability and somewhat less direct inform- 
ation in the areas of Motivation to Accept innovation and Implementation Out- 
comes (although in this last area it may have predictive value). The kinds 
of information yielded by the existing measure, along wUh the uses for such 
information, are examined in a more specific and detailed manner in the next 
section • 
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AN APPLICATION OF THE DISCREPANCY MEASURE TO IMPLEMENTATION ASSESSMENT 

The discrepancy measure was designed exclusively for purposes of 
examining anc then segmenting the market for Innovative educational products. 
Using the measure, data were collected In reference to three specific 
educational products. An analysis of the results of this application 1s 
presented In another report (Sikorski and Hutchlns, 1974). 

Although the measure was not designed to assess Implementation of the 
three products, In practice It was found to elicit data regarding Implementation. 
Often, along with Identifying the discrepancies for a product in his school, 
a subject would volunteer information about the way these problems had affected 
implementation, whether by impeding use in some way or by giving rise to 
adaptive solutions. This unforeseen finding generated the present examination 
of the measure as a means of assessing Implementation. In this section we 
report the results of a tentative and exploratory analysis of the marketing 
data to determine what Information it contained regarding Implementation. 

The information volunteered by subjects indicated a variety of possible 
responses to discrepancies; these were classified into eight categories, 
five of them positive (e.g , Staff Substitution, or the recruitment of volunteers 
when paraprofessionals could not be hired), and three of them negative (e.g., 
Implementation Delay). Responses in each category can be associated with 
particular kinds of discrepancies. While the set of response categories may 
be far from exhaustive, it does present a classification scheme which can be 
helpful to others engaged in implementation assessment. And with further 
refinement, the associations between particular discrepancies and particular 
patterns of Implementation can provide important guidance for the marketer. 
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Knowing what kind of accomodation can be made to a discrepancy, he can find 
a way to offer the most appropriate help to potential adopters who are 
experiencing that discrepancy. Thus the early adopters of a product will 
generate suggested procedures that later adopters can follow In order to 
Increase their chances of successful Implementation; such procedures can be 
packaged In some way as part of the marketing strategies designed for different 
segments of the market. 

LJ-^cause the existing data were not collected for the purpose of assessing 
Implementation, however, the data content amenable to analysis for this purpose 
was limited. The fact that a discrepancy exists does not necessarily reveal 
anything about Its effect on the Implementation process. A discrepancy state- 
ment alone Is a measure of Implementation only in the simplest cases, when 
the nature of the discrepancy clearly Implies either that It effectively 
blocked Implementation or that It was so minor that a successful adaptation 
undoubtedly occurred and allowed Implementation to proceed with little delay. 
In other cases the statement of discrepancy by Itself Is merely a statement 
of a problem or difficulty with no indication of what effect, if any, that 
problem had on the course of Implementation. (Ultimately, through analyses 
of the kind reported here, it could be learned what accomodations are likely 
to be prompted by a given discrepancy.) 

For purposes of implementation assessment, then, a statement of discrepancy 
is not complete without an accompanying statement describing the result or 
effect of the discrepancy on the Implementation process. Such statements take 
the form of a description of the response to the discrepancy, whether this 
response is successful or not. Some descriptions may be of failures to 
implement because of the discrepancy, others may be of successful adaptations 
and continued implementation. 
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Fortunately, in the present study the information that subjects 
provided on discrepancies was frequently accompanied by information regard- 
ing their responses to discrepancies. (This fact, incidentally, may offer 
some further support for the logic of tying the discrepancy approach to 
implementation assessment). But because interviewers were not formally 
seeking data on implementation, the data were not collected in a systematic 
way or a standard format. As a result the levels of completeness and 
comparability in the data do not permit a detailed, quantified analysis. 
The data are instead treated as partial case studies and are analyzed in 
an informal way in terms of categories of implementation response to 
discrepancies. 

Eight categories of implementation response have been determined. The 
first five of these are positive responses, varieties of adaptation to the 
discrepancy challenges. These include Staff Substitution, Materials Augmen- 
tation Equipment Substitution, Administrative Adaptation, and Funding Improvi- 
sation. The last three categories of implementation response are predominately 
negative in character, subsuming cases where implementation was partially or 
completely impeded as a result of discrepancies. These include Implementation 
Delay, Partial Implementation, and Potential Abandonment. 

The discussion which follows concerns the implementation of three specific 
educational innovations. Two of these innovations are products consisting of 
instructional materials, while the third is more of an educational process. 
These three innovations are described in detail in Appendix B of this report, 
but it is necessary to note here that inherent differences among products 
produce differences in the nature and scope of the efforts required for their 
Implementation. Minicourses, developed by Far West Laboratory for Educational 
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Research and Development, are multimedia, self-instructional courses for 
pre- or inservice teacher training, and students are only peripherally 
involved in their implementation. Man: A Course of Study (MACOS), developed 
by Education Development Center and distributed by Curriculum Development 
Associates, is a set of multimedia student materials designed to teach social 
science skills of observation, hypothesizing, and problem solving. Multiunit 
School/Individually Guided Education (IGE), developed by Wisconsin Research 
and Development Center for Cognitive Learning and Institute for the Development 
of Educational Activities (I/D/E/A), is essentially an organizational system 
that replaces traditional, self-contained classrooms with larger, nongraded 
units. 

IGE was the only innovation to produce examples in this analysis for 
the first implementation response category. Staff Substitution. Most of 
the responses were to the discrepancy labeled Personnel. This discrepancy 
arose because IGE provided for centralization and specialization of teaching 
staff and the freeing of teacher time by hiring teachers' aides and secretaries 
to handle supervisory and clerical duties. The problem was that many of the 
schools implementing IGE found that they could not afford to hire these extra 
staff members, or that they were not permitted to hire them because the school 
civil service system did not include these positions. The most common adaptive 
response in implementation was to substitute parent volunteers to perform the 
duties of teachers' aides. One fortunate school was able to acquire sufficient 
funds to hire students from a local college to perform the duties on a part- 
time basis, in this way providing a more permanent solution to the problem. 

Less fortunate implementation efforts were forced to adapt to the 
discrepancy in a less satisfactory way, however. These schools, apparently 
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unable to obtain even parent volunteers, resolved the problem by asking 
teachers to perform all of the duties. Since I6E In fact Increases the 
clerical workload because of Its program of extensive testing, this 
resolution placed additional duties on teachers. In some cases the piling 
up of duties lessened the enthusiasm and cooperation in the implementation 
effort on the part of teachers. It is possible that with advance warning 
and suggestions from an astute marketer, these schools would have managed 
to resolve their staffing problem in some more effective way. 

Another example of a response included in the Staff Substitution 
category is a case in which the implementors did not want to set up a 
hierarchy among teams of teachers as provided by the I6E plan. A team cooper- 
ative effort was felt to be more functional in that particular implementation 
setting, and so the position of chief was rotated among all staff members in 
each of the teacher groups so that pay levels and responsibilities were equalized. 

In the second implementation response category. Materials Augmentation, 
were coded responses associated with discrepancies labeled Adaptability, 
Completeness, and Materials. In one instance a school implementing IGE found 
a need for a guide to the reading materials used in its state so that it could 
use some of the Instructional materials provided by IGE. The school was able 
to borrow the needed reference work from a neighboring school district. In 
two other instances, one Involving Mini courses and the other MACOS, the subject 
matter coverage supplied by developers was felt by Implementors to be incomplete. 
The adaptation in each of these implementation efforts was to supplement the 
product with materials of the desired type from other sources. In a final 
Instance there was simply a delay in obtaining materials to be used in IGE 
implementation, and in this case other materials were substituted in place 
of those which were unavailable. 
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In the third implementation response category, Equipment Substitution, 
were coded responses associated primarily with the discrepancy labeled Materials, 
which was defined as concerning the availability of materials and equipment 
necessary to use an innovative product. Most of the instances in this category 
involved the implementation of Mini courses, which require the use of videotape 
equipment for complete implementation. Some schools simply did not have access 
to videotape equipment and were forced to substitute audiotape recorders instead; 
in one case, personnel brought in such equipment from their homes. In another 
case a supervisor "substituted" for videotape equipment, observing teacher 
performance in practice teaching and then providing feedback. Also placed in 
this response category was an adaptation associated with MACOS which involved 
converting 8mm films to 16mm so that the materials supplied by the developer 
would be compatible with the equipment owned by the school. 

It is easy to see the value of this kind of information for development 
and dissemination. Since the developers of Mini courses state that the product 
can be effective when used with audiotape instead of videotape, this adaptation 
could be publicized in dissemination efforts targeted to schools that have a 
Materials discrepancy. In this way, the dissemination program could promote 
a solution that is likely to lead to effective implementation-'-as contrasted 
with solutions like the supervisor's "substitution" for the VTR, which might 
tend to defeat the Minicourses' emphasis on self-instruction for teachers. 

Associated with the fourth implementation response category. Administrative 
Adaptation, were such discrepancy labels as Disruptiveness and Administrative 
Requirements. Two instances involved problems of scheduling people. The 
first was associated with implementation of IGE, which requires the combination 
of several different grade levels into a single group. In one state, different 
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hours of attendance were required for the different grade levels. This gave 
rise to complex and time-consuming management of attendance schedules. The 
second instance^ associated with Minicourses, involved the difficulty of arrang- 
ing times for teachers to leave their classrooms so they could participate 
in the course. The adaptation was that teachers developed closer cooperative 
working relationships so that an adequate schedule could be arranged. 

Two other instances associated with Administrative Adaptation involved 
the scheduling of things. One was associated with the implementation of IGE 
and the other with the implementation of MACOS. Both cases of adaptation were 
necessitated by the sharing of instructional materials and supplies among 
schools within a single district. The adaptation was to modify administrative 
arrangements for such sharing until the shared materials were efficiently 
distributed. 

There was a straightforward relationship between the fifth implementation 
response category » Funding Improvisation, and the discrepancy labeled Cost. 
There was an instance in which a school wished to implement MACOS but could 
not afford the product. The adaptation in this case was to share the cost 
among a number of schools in the district. In another instance a school 
implementing IGE could not afford to hire teachers' aides, but it managed to 
obtain a special grant from an outside source for this purpose. Outside 
funding was also the adaptation of a school implementing the Minicourse. The 
money was used to pay for released time for teachers to take the course. Again, 
the actions taken by these adopters could be passed on, through dissemination 
efforts, to other schools. 

The sixth implementation response category. Implementation Delay, 
contained the largest number of cases and was associated with the greatest 
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variety of discrepancy labels. Virtually any kind of aiscrepancy could 
lead to delay. Cases assigned to this category lacked the positive, active 
adaptations which characterized cases assigned to the first five categories, 
and they also lacked some of the negative aspects of cases assigned to the 
seventh and eighth categories. Partial Implementation and Potential Abandonment. 
Discrepancies which merely caused implementation delays were those which were 
too severe for easy adaptation but too mild to block implementation permanently. 

One group of cases assigned to the Implementation Delay category was 
associated with implementation of I6E. These cases arose because I6E is a 
relatively drastic restructuring of school organization, and, as such, it 
quite frequently engenders staff opposition. This problem was assigned the 
discrepancy labels Administrative Requirements, Reorganization, and Disruptlve- 
ness, and the response to the discrepancy was a slowing of the implementation 
process. This delay allowed the additional time necessary for ironing out 
difficulties and making the required role adjustments. There was time for 
constructive interaction between those teachers in favor of the innovation 
and those opposed. In some cases resistant teachers were gradually convinced 
of the merit of the innovation, while in others there was time for dissident 
staff members to leave the Implementation site. 

Delay as an adaptive response was also observed in implementation efforts 
when parental or community opposition to IGE surfaced. The discrepancy labeled 
Priority encompassed such instances of outside pressure. Slower adoption of 
the innovation in these cases allowed time for the gradual growth of the 
community support for the new program. 
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Delay In response to opposition from one or more of the constituencies 
of the innovation can serve a constructive purpose* but the same cannot be 
said for delay as a result of equipment problems. The delay may be no less 
necessary when occasioned by equipment problems, but it serves no heaiing 
purpose. Equipment problems labeled Materials discrepancies were most often 
encountered in association with the implementation of Minicourses. For 
example, in one case delay was caused because the videotape equipment kept 
breaking down. In another case, where the discrepancy was labeled Training, 
it was found that more time was necessary for training in the use of videotape 
equipment than had been estimated. In still another case delay was caused 
because no adequate space could be found in which to use the videotape equip- 
ment, a problem which was termed "Space." Equipment difficulties also arose 
in association with MACOS when implementation was slowed because a school 
could not obtain enough of the required film projectors. Since this kind of 
delay has no beneficial effects for adopters, the marketer should be able 
to anticipate the equipment problems that may befall schools, and the marketing 
"treatments" should include providing strategies for overcoming these problems. 

Some final implementation responses which were coded in the Implementation 
Delay category were associated with Training and Priority discrepancy labels. 
These included cases where the existing school arrangements or plans were 
in some way inappropriate to implementing the innovation. One example, associated 
with implementation of IGE, involved a school in which teacher training for 
the innovation was conducted at the end of the school year prior to implementation. 
The teachers were exhausted in June, when the training took place, then after 
the summer recess they had forgotten much of what they had learned in the training. 
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This lack of adequate training caused the implementation process to go more 
slowly the following year. Delay was caused in another instance in which 
the implementation of IGE interfered with another complex innovation that 
had higher priority and was being implemented concurrently in the school. 

The seventh implementation response category. Partial Implementation, 
is associated with discrepancies for which no other adaptation could be 
found than abandonment of a portion of the implementation effort. In one 
case, associated with the Personnel discrepancy code, implementors thought 
that the IGE program had not adequately specified the personnel required 
or their qualifications. During implementation it was discovered that the 
program had not been properly staffed, and the parts of the program dependent 
upon staff with certain qualifications had to be abandoned in the absence 
of the required staff. In another case there was a Theory discrepancy; 
implementors did not completely accept the rationale for IGE, and consequently 
they selectively implemented only those portions of the program compatible 
with the theory they endorsed. 

In a similar case, associated with the Priority discrepancy label, it 
was found that implementation of MACOS conflicted with other innovations of 
higher priority for the district. The scope of the implementation effort 
was reduced, and MACOS was implemented on an individual basis by only a 
few enthusiastic teachers rather than by the district as a whole. 

The eighth and last implementation response category. Potential Abandonment, 
includes cases in which an attempt had been made to implement the innovation, 
experience had indicated that discrepancies existed for which no long-term 
adaptation appeared adequate, and consequently abandonment of the implementation 
effort was being seriously considered. Several of these cases involved imple- 
nientation of Minicourses. These were associated with such discrepancy labels 
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as Target, Subject Matter, Time Required, and Disruptiveness . In these 
cases teachers either resisted exposing themselves to criticism, believed 
the course level was too low or the subject matter not complete) enough, 
thought too much time was required for the benefits received, or resented 
being absent from their classrooms in order to take the course. In each 
case it appeared that opposition was great enough to prevent further use of 
Minicourses in these implementation settings. 

One other case of a Potential Abandonment implementation response was 
found in an implementation of IGE when a Developer discrepancy was present. 
In this case the implementors did not respect or trust Uie source of the program, 
and so they were suspicious of receiving any benefits from implementation. 
This attitude generated such a half-hearted and tentative Implementation process 
that it appeared likely the effort would be abandoned. 

The analysis reported in this section of the paper has been limited by 
the unsystematic nature of the data available. Nevertheless, even this relatively 
crude analysis has ind1c2t<»d that the discrepancy measure is capable of becoming 
a potentially rich and varied source of data for assessing the implementation 
of innovation in education. Through the device of uncovering discrepancies 
which may signal Implementation problems, it also elicits Information about 
the way the problems are resolved. This information not only Increases the 
general fund of knowledge about the process of innovation implementation but 
also adds an important dimension to the work of dissemination. Based upon 
the experience of users of an innovative product, the later adopters can be 
enabled to make appropriate adaptations to Improve their own implementation of 
the product. The next section of the report considers modifications that 
could be made in the measure to increase its effectiveness of implementation 
assessment. 
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MODIFICATION OF THE DISCREPANCY MEASURE FOR IMPLEMENTATION ASSESSMENT 

In its present form the discrepancy measure has certain limitations for 
purposes of its application to Implementation assessment. Some of these 
limitations were discussed explicitly above, while others were barely touched 
upon. In this section the limitations of the measure for Implementation 
assessment are considered more directly, and some potential modifications 
that might be made to overcome the limitations are discussed. 

Without compromising the value of the measure for Its original purpose 
of educational market analysis, there are a number of modifications that can 
be considered. One has already been discussed. This Is to Include a specific 
request for a descriptive statement concerning the adaptation or response to 
each discrepancy. The analysis has Indicated that a great deal of Information 
regarding adaptation to discrepancies happened to be collected In this study, 
but any modification of the existing measure for implementation assessment 
would Include provision for more systematic collection of data of this nature. 

In this section are suggestions for revising the measure in accordance 
with the critical incident technique, which offers a method of collecting 
data on complex topics from concerned observers, and which the present measure 
already resembles to some extent. In addition to exploring this methodological 
issue, the section suggests other dimensions of Implementation assessment that 
could enlarge the measure's scope of Inquiry. These Include the timing of 
adaptation, the severity of the discrepancy, the central ity or radlcalness of 
the adaptation, and the generality of the need for adaptation. With these 
modifications, the discrepancy measure could become not only a more valuable 
tool for examining Implementation in schools but also, as explained below, an 
aid In generating still other measures. 
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Methodological Considerations 

One of the present limitations of the discrepancy measure for implementation 
is inherent and therefore not subject to modification. This limitation is that 
the measure is primarily directed towards negative dimensions of implementation, 
that is, those implementation patterns that arise in response to problems. In 
the first section of this chapter, the ability of the discrepancy measure to 
assess negative aspects of implementation was mentioned as one of its major 
potential strengths. Paradoxically, the measure's major strength is also one 
of its major weaknesses because in its emphasis on the negative the measure 
tends to ignore positive aspects of implementation. But this lack of balance 
In the discrepancy measure would be a problem only if the measure were the only 
one applied to a given instance of implementation assessment. When used in 
conjunction with other measures, its emphasis could be balanced by the strength 
of other measure? that assess positive aspects. 

Other limitations of the discrepancy measure for implementation assessment 
are not inherent, however, and further development and modification of its 
methodological characteristics could overcome many of them. From a methodolo- 
gical point of view, in its application to implementation assessment the discrepancy 
measure can be seen as a variant of a measure known as the critical incident 
technique. For this reason information regarding the development and systemat- 
ization of the critical incident technique can be used as a context for discuss- 
ing the further methodological development of the discrepancy measure for 
purposes of implementation assessment. 

The critical incident technique is old and quite basic to social science 
but was given a definitive specification by Flanagan (1954). The critical 
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Incident technique does not consist of a single rigid set of rules governing 
data collection. It Is rather a flexible set of principles which must be 
modified and adapted to meet the specific situation at hand, In this case 
the assessment of Implementation. For any activity to be studied, knowledge- 
able persons are asked to describe observed behavior relevant to that activity, 
for example. Incidents of effective or Ineffective performance In a particular 
job. In this case the activity Is Innovation Implementation, and the critical 
Incidents are actions by school staff members In the Implementation process. 
The essence of the teti.nlque Is that only simple judgments are required of 
observers, reports are obtained only from qualified observers, and all observations 
are evaluated by the observer In terms of an agreed-upon statement of the purpose 
of the activity. The critical Incidents reported are based exclusively on 
observable behavior, rather than Interpretations or opinions based on general 
Impressions, and the behaviors are those which are significant In terms of the 
activity being assessed. In short, the critical Incident technlqut produces, 
first, a record of specific behaviors collected from those In the best position 
to make the necessary observations and evaluations, and second, an analysis of 
these observations. 

Application of the critical Incident technique In the context of Imple- 
mentation assessment would be analogous to some of Its prior uses. It has been 
used to study operating procedures— for Instance, what pilots do to avoid accidents 
In flying— with a view to Improving the effectiveness and efficiency of these 
procedures. In a related application, the critical Incident technique has been 
used to Improve equlpinent design; after collecting and studying reported problems 
with aircraft Instruments, researchers were able to recommend changes In the 
equipment. It seems clear that the technique could shed light on the operations 
that go Into Innovation Implementation In schools and contribute to the design 
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of more readily implemented products. observers in this case would 
be implementors, including district adm .ration officials, principals 
and teachers. The observers would be r ting incidents of adaptive 
behavior in response to one or more dis ancies which emerged during the 
course of an implementation effort. 

Basically five steps would be taken in urder to design and use a discrepancy- 
adaptation measure as a critical incident technique in implementation assessment. 
The first would be to determine the goals of implementation, which is the activity 
for which the critical Incidents would be formulated. The second would be to 
develop plans and specifications for collecting the actual discrepancy and 
adaptation incidents. This would include drafting clear instructions for persons 
acting as observers in reporting the critical incidents. The third step would 
be to collect the critical incident data, either by personal 
administered questionnaire, in a more systematic manner than in the present 
study. The fourth step would be to analyze the data, which would involve 
establishing categories and codes for the discrepancy and adaptation incidents 
like those presented in the previous section of this report. The fifth step 
would be to interpret and report the critical incidents identified in imple- 
mentation. 

In this progression several methodological problems would have to be 
overcome. The criteria to be used in reporting the discrete instances of 
behavior must be made clear to observers. In some cases the implementation 
process may be very complex and confusing even to those participating. There 
might even be situations in which observers would need some training before 
they became qualified to collect appropriate incidents. 
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A closely related problem would be the selection of one or more levels 
along the specifity-generality continuum for use in reporting the adaptation 
incidents, both in the process of collecting the incidents and in later phases 
of analysis and final reporting. That is, it should be determined whether 
the type of incident sought would be "a radio advertisement asking for parent 
volunteers was broadcast," or "parents were brought in." A final methodolo- 
gical problem to be overcome would be the inductive development, during analysis, 
of a set of major area and subarea headings in devi3ing a classification scheme 
for the implementation adaptation incidents. The classification scheme devised 
would have to be appropriate both to the implementation setting and to the needs 
of the assessment, bearing in mind its application in the study of implementation 
and in market analysis. 

If it is developed in this way, to reflect more directly the critical 
incident technique, the discrepancy measure might also serve the ends of 
implementation assessment by actually helping in the development of other 
measures. In many educational settings measurement instruments are unsatis- 
factory in part because the behaviors they purport to measure are specified 
a^ priori (Mayhew, 1956). A priori specification may be a particularly dangerous 
procedure in the assessment of implementation, because it is often difficult 
to determine beforehand in what directions the implementation process will move. 
The critical incident technique is of potential value in implementation assess- 
ment precisely because the incidents are not categorized until after they have 
been collected. But beyond that, such a measure is of potential value because 
it provides empirically derived classifications of behavior which can then be 
used either as a framework for subsequent measurement or as the materials out 
of which other instruments can be developed. 
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An adequate collection of critical adaptation Incidents could place 
categories of Implementation behavior on an empirical base, providing for 
greater validity In any subsequent measurement Instrument. If enough 
critical Incidents could be collected, reasonably complete categories of 
effective and Ineffective Implementation behavior could be derived. Of 
course It must be remembered that a discrepancy measure used alone will 
only yield Incidents that are associated with discrepancies, or "snags" 
in the implementation process. However, as discussed above, two or more 
complementary measures can be employed to assess a more complete array of 
behaviors. The derived categories of behavior could be used either for a 
check list, a rating scale, or one axis of an objective test in subsequent 
Instruments. The critical incidents of adaptation could also serve as a 
general source of empirically derived raw material out of which subsequent 
specific questionnaire items could be drafted. This would meet a need for 
assessing the implementation situation in words and concepts that are meaning- 
ful to the implementors. 

Content Considerations 

Although this paper has Indicated that the discrepancy measure yields 
rich data for implementation assessment, it has also acknowledged that the 
measure in its present form does not encompass all dimensions of potential 
Interest in Implementation assessment. This portion of the report discusses 
some of the additional dimensions which might be added as the discrepancy 
measure is modified for purposes of implementation assessment. These include 
the time of the discrepancy and/or the adaptation, the severity of the 
discrepancy, the central ity of the adaptation, and the generality of the need 
for adaptation. Such refinements would be added by providing instructions 
to observers, or structuring questions in interviews or questionnaires to 
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include additional dimensions of the implementation process. 

For purposes of implementation assessment, one feature currently lacking 
in the discrepancy measure is the dimension of time. Implementation is an 
attempt to change behaviors in some way, so an assessment of implementation 
must as a basic step assess whether a change has taken place. One means of 
doing so is to administer a pretest measure to collect data to compare with 
outcomes obtained with a posttest measure. A more sophisticated approach 
would obtain one or more interim measures as well so that a more sensitive 
assessment of the entire process of implementation could be made. 

But as a marketing analysis technique the discrepancy measure would 
usually be administered only once in a given field setting. For implementation 
assessment purposes, then, it would be important for the measure to identify 
and distinguish among the various time contexts in which the discrepancies and 
adaptation incidents could be reported. A discrepancy may exist at any time 
during the implementation process. The measure would need to determine the 
relationship between the time the discrepancy arises and an adaptation is 
made in response to it (if this occurs), and the time of report. The discrepancy 
and/or adaptation may be reported in an anticipatory time context, a concurrent 
time context, or a retrospective time context. Each time context of report 
has its own particular bias, and any analysis or interpretation of findings 
would need to be cognizant of these biases. An anticipatory time context may 
produce bias because it is not always possible to project accurately and clearly. 
A concurrent time context, because of the observer's proximity to the situatioOv 
may produce such biases as an exaggeration of the severity of the situation. 
A retrospective time context probably produces the greatest accuracy of report, 
but factors of selective retention can produce distortion through passage of 
time. 
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Another content modification that might be Introduced Into the discrepancy 
measure for purposes of Implementation assessment would be a more detailed 
determination of the degree of severity of each discrepancy. As presently 
constituted, the discrepancy measure does assess degree of severity on a 
three-point scale, but an assessment of implementation would benefit from 
a somewhat different kind of determination, covering not just the perceived 
severity of the discrepancy as It arises, but also the feasibility of making 
an adequate and lasting accomodation to the discrepancy. A problem which 
originally looms as a formidable one may in fact be very effectively overcome, 
and the measure should provide an accurate record of such a case. 

Thus, one way of determining degree of severity would simply be to 
collect more information regarding the discrepancy and its corresponding 
adaptation. This information could then be used to construct a more complex 
scale during data analysis. One dimension along which such a scale could be 
constructed is the degree of permanence of the most feasible adaptation. 
Severe discrepancy might well be indicated even in a situation in which an 
adequate adaptation had been formulated, if the adaptation was adequate for 
only a limited period of time. An example of such a situation is that in which 
necessary funding can be obtained only on a temporary basis. End points along 
this scalar dimension might be a discrepancy for which an adequate and 
permanent adaptation can be formulated, and at the other end of the scale, a 
discrepancy for which no adaptation is adequate on even a temporary basis. 

More information would need to be collected so that other distinctions 
among adaptations could be made during analysis. One Important dis.tinction 
is a determination of the "central ity" of an adaptation with respect to the 
innovation as it was designed. In adapting an innovation to "save" it and 
make it feasible for their environment, implementors may move the Innovation 
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through a single drastic adaptation or a series of small adaptations so that 
It loses salient features of the original design. In moving away from the 
model as originally defined to construct a model of their own through 
adaptation, the implementors may so radically alter the Innovation that It 
can no longer be considered the same Innovation. Such an Implementation 
process, while It. might be considered a success by the Implementors, could 
be viewed as a failure by the developers of the Innovation who see a result 
so far from their Intention. Therefore, Information pertaining to this 
central ity dimension in implementation assessment can have Important meaning 
for developers as well as market analysts. When radical adaptation tends 
to occur, the marketer can try to devise treatments that will give adopters 
other options that are more faithful to the original design. And if this 
proves ineffective, the developer might return to the drawing board and 
produce a product that can satisfy both his conception and users' apparent 
preferences. 

In case developers are inclined to be so flexible, the data should permit 
a distinction between a general adaptation and an idiosyncratic adaptation, 
or a determination of where the burden of adaptation should lie. The term 
"burden of adaptation" refers to the locus of responsibility for changing the 
design of an innovation in response to implementation problems. In the case 
of an idiosyncratic adaptation the responsibility lies with the Implementors. 
The designers of an innovation cannot be expected to have made their product 
amenable to all particular implementation settings. But they can be expected 
to have made their product adaptable to common kinds of settings, and in the 
case of a general adaptation the burden of adaptation shifts to them. The 
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ability to make distinctions of this nature in data collected by means of 
the discrepancy measure might allow an implementation assessment to explain, 
for example, whether a case of failure to implement was primarily the "fault" 
of the innovative product or of the school. Such an analysis would, of course, 
be highly meaningful for developers, dissemination specialists, and school 
personnel. It would also have to be conducted very carefully, so as not to 
draw any unfair and injurious conclusions. 

This section of the report has described a few dimensions which could 
enhance the utility of the discrepancy measure in implementation assessment. 
By including considerations of time, severity, central ity, and generality, 
the measure could yield a more comprehensive picture of implementation which 
would have Important uses for the marketer; yet these modifications would not 
make the measure very much more complex to develop or administer. 
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CONCLUSION 

This report has explored the possibility of using the discrepancy 
measure t^ft^ assess the Implementation of educational Innovations. The 
data thus generated would have value not only In the general field of 
implementation assessment, but also In the context of dissemination. The 
marketer, knowing how adopters have behaved when confronted with the challenge 
of particular discrepancies, is in a better position to deal with the non- 
adopters who experience the same discrepancies. He can try to design a 
marketing campaign that will, lead to adoption and, beyond that, to effective 
use of the innovation through constructive adaptation. 

The first section of this report considered the discrepancy measure 
in the general context of implementation assessment and indicated that the 
measure makes a potential methodological contribution in this area. Further- 
more, the measure provides or could provide data on virtually any implementation 
variable. An analysis of existing data, in the second section of the report, 
indicated that even in its present form the measure generates relatively rich 
data on implementation and permits a rough classification of different imple- 
mentation behaviors. A number of modifications which could be made in the 
measure to make it even more suitable for the collection of implementation 
data were discussed in the third section of the report. 

Although problems remain in attempting to use the discrepancy measure 
in the new area of application, certain unique strengths of the measure in 
the context of implementation assessment indicate the potential value of this 



exploration. Valuable data concerning Implementation could be collected at 
no greatly added cost 1n the course of marketing analysis, thus adding 
another function to the discrepancy measure and a further dimension to the 
analysis. This and other considerations lead to the conclusion that the 
discrepancy measure's potential In the area of implementation assessment 
merits continued exploration. 



REFERENCES 



Flanagan, J.C. The critical Incident technique. Psychological Bulletin , 
1954, 51, 327-358. 

Goodlad, J., and Klein, M. Behind the classroom door . Worthlngton, Ohio: 
Jones Publishing Company, 1970. 

Gross, N., Giacqulnta J., and Bernstein, M. Implementing organizational 
innovations: A sociological analysis of planned educational change . 
New York: Basic Books, 1971. 

Mayhew, L.B. The critical Incident technique In educational evaluation . 
Journal of Educational Researr ':, 1956, 49, 591-598. 

Sikorski, L.A, , and Hutchlns, C.L. A study of the feasibility of market ng 
p rogramming for educational R&D products . San Francisco: Far West 
Laboratory for Educational Research and Development, July 1974. 



A-1 



Appendix A: 
DISCREPANCY CODE WORD DEFINITIONS 

The following Is a list of 34 potential discrepancies used to characterize 
schools. Each Is presented with a sample test Item. Items will vary depending 
on the nature of the product and the ways discrepancies may occur. For example, 
the Item under "Sequential" will be stated differently when it is in reference 
to a curriculum which is not. sequential from the way it is stated to detect 
discrepancies with MACOS, which is sequential. Also, they will usually be stated 
differently for adopters and non-adopters. For one thing, adopters cannot give 
a response which is scored "3" (discrepancy would prevent adoption). The sample 
items listed here represent a variety of circumstances and Include adopter and 
non-adopter versions, and versions for the three different products. For example, 
the item for "Adaptability" is the version which would be used with a MACOS 
adopter; the item for "Administrative Requirements" is the version which would 
be used with an IGE control (non-adopter). 

Adaptability -- Degree to which product is adaptable; amenable to revision for 
specific purposes of purchaser. 

Did you or others at this school feel that the materials or methods of Man; A 
Course of Study would have to be changed (i.e. modified or adapted) to better 
fit your needs? 

^Yes 

tio 

Did you perceive that it was, in fact, sufficiently adaptable, i.e. could you 
easily change it in the ways you deemed necessary? 

'Ip.s 

No, but this wasn't Important to us. 

No, it seemed somewhat unsatisfactory in this regard. 

No, it was extremely unsatisfactory in this regard. We almost did not adopt 

it for this reason. 
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Administrat ive Requirements -- Adequacy of present administrative capability for 
implementing product. 

Consider a teaching system which requires the following kind of administrative 
arrangement: an organizational system that replaces traditional, self- 
contained classrooms with larger, nongraded units. In each unit, a unit 
leader, two or three staff teachers, a first year teacher, a teacher aide, 
an instructional secretary, and an Intern work with 100-150 students in a 
three or four-year age span. Unit leaders and building principal make up 
an instructional improvement committee and define the school's goals. At 
the district level, a systenwide policy committee (central office administrators 
and consultants, principals, unit leaders and teachers) develops policy 
guidelines and coordinates the use of human and physical resources. 

Thinking about this school, and the decision makers here, do you think 
there would be any specia l problems converting to this arrangement (aside 
from the obvious problem that this is a major move which would require 
careful consideration)? 

I don't think our school would have any special problems not 

faced by any other school . 

We might have more difficulty than other schools would. 

We would definitely have much more difficulty using this arrangement. 



We could never implement this arrangement here. 



Ba ndwagon — Extent of and importance of knowledge of others' use of product. 



When you adopted Man: A Course of Study , did you know of other schools which 
had adopted it before you did? 

Yes- 

Uo 

Wis this a factor in your decision to use the program? 
No 



Ycs« it influenced us to adopt. 

Yes, it was the basis of some of our doubts about the program. 



Yes, it almost kept us frotn deciding to use the program. 
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Complete/Supplementary -- Desirability of Complete (v. Supplementary) - 
Curriculum. 

Which of the following best describes the scope of elementary (5th-6th grade) 
social studies materials this school could use now? 

We need only supplementary material s--we definitely would not adopt a complete 

program at this time. 

We need only supplementary materials—we probably would not adopt a complete 



program at this time. 

We need only supplementary materials, but we would also consider a complete 

program. 

We need a complete program. 

Core/Elective - Degree to which product fits In with existing curriculum or 
other structure. 

Uhich of the following statements is an accurate description about the 
teaching of elementary (5th-6th grade) social studies at this school? 

This has been well-establ Ished In the school's schedule of activities. 
^D-ffinlte times and places are set aside for regular lessons. 

Elementary (5th-6th grade) social studies has not been regularly 
^scheduled, but it has been carried out on an irregular basis. 



Jhls hcis not been taught at this school, even on an irregular basis. 

This hc!.: not t)':i.n taught and t\Mro. is resistance to teaching it. 



Cost - Availibility of financial resources for purchasing and implementing 
product. 

used year after year with all elementary teachers. Does this price seem 
reasonable? 

Our school could easily work out some way to manage this. 

Our school could work out a way to manage this, but not easily. 

Our school probably couldn't work out a way to manage this. 

We could not afford this under any circumstances. 
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Demonstratability Degree to v/hich product can be demonstrated prior to purchase. 

Was it required for someone at the school to see Man; A Course of Study demon- 
strated before the school could decide to use it? 

Yes 



No 



Was it easy to arrange to see a demonstration? 
Yes 



No, this was a bit of a problem. 



No, this was a major problem. 



Did you see a demonstration? 
No 



Was it satisfactory? (Check one) Yes; No, it was somewhat unsatisfac 

tory; ^No, it was highly unsatisfactory. 



Devel oper -- Degree of respect or trust in developer of product. 

Which of the fo>l owing is the feeling in this school about the Far West Laboratory 
for Educational Research and Development? 

never heard of it--so I really can't say. My predisposition is to figure 

it's a reputable organization. 

It's a reputable organization; and the products it develops are high-quality. 

We have our doubts about products developed by the FWL, but most are probably OK. 



We don't trust FWL products. 



In this school, FWL products would not be adopted. 



Disruptiveness -- Degree to which product creates or stimulates latent or existing 
problems such as threatening teacher or student discipline. 

Consider that an elementary (5th-6th grade) social studies program is a complete, 
year-long program which teaches cultural anthropology and uses an inquiry approach. 
Such a program may be disruptive in various ways. For example, it may displace 
other activities, it may disrupt teacher-teacher or teacher-student relationships, 
it may disrupt the day-to-day schedule of activities, etc. In this school, do 
your feel such a program would be disruptive in those or other ways? 
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Ho 

r^aybe, but I wouldn't predict that there would be very extensive disruption. 

Yes, I'd expect this to be very disruptive In some or all of those ways. 

^Yes, this would be extremely disruptive. 

Divisibility Degree to which product can be used in part. 

In this school, is it the usual practice to use just part of a new program, on 
a limited or trial basis, before you actually decide to adopt the entire program? 

UOt we don't usually do this. 

We like to do this, but we frequently make exceptions. 

Yes, this is our usual practice; we almost never make exceptions. 

Yes, this is our practice. We never make exceptions. 

Early/Late Feedback -- Appropriateness of speed with which information regarding 
effect of product (i.e. evaluation data) is obtainable. 

When a new product or program is adopted by this school, at what point in time 
is it best to actually be able to see the results? 

We don't need to see such early results—we can wait a year for that. 

We like to see results before the end of a year of use, but this is not 

necessary. 

We want early knowledge of results; after 3 or 4 months of use. 

We need quick feedback--we have to see results after a month or so of use. 

Effect -- Appropriateness of major effect or objective of product. 

Consider an elem'entary (5th-6th grade) social studies program which Increases 
"ability to reason" and knowledge of cultural anthropology. Would this kind of 
program be adopted at this school? 

^This is the type of effect we would want in a 5th-6th grade social studies 

program. 

We want some other thing (s) not offered by this kind of program, but it's 
basically what we would be looking for. 

We would hope to find something other than this to use here. 

Q We wouldn't adopt a course like this. 
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Evaluation Acceptability and Importance of evaluation data regarding product , 
presented by developer. 

When considering a new curriculum, what kind of evaluation results does this 
school need to see before you can decide? 

We don't need to consider evaluation results. 



We need to see teacher and student changes documented in whatever way the 
"evaluator chooses. 

Usually, we need to see results on standardized tests. 
JWe almost always need to see results on standardized tests. 
We always need to see results on standardized tests. 



External Support - Feasibility of outside (e.g. federal) funding sources to 
support purchase and implementation of product. 

In some schools, there is the chance to get outside funding support (e.g. Title 
I, III) for adopting inservice training materials. What is the situation In 
this school? 

We wouldn't need any outside funding for this here. 

We could easily get the outside funding support necessary for this kind of 

adoption. 

We probably could not get the outside support we'd like, although this would 

not prevent us from adopting. 

We probably could not get the outside support we'd like, and this would 

probably prevent us from adopting. 

We couldn't get outside support for this, and this fact keeps us from adopting. 



Horizontal/Vertical Appropriateness of design of product for one grade level or 
for successive grade levels. 

Which of the following is an accurate statement about the kind of 5th-6th grade 
social studies program that would be adopted here? 

We want a social studies program tailored specifically to. those grade levels. 

Right now, we have a slight preference for a program which is usable for all 
or most other social studies classes at other grade levels. 
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We have a definite preference for a program which cuts across all (or most) 

of our grade levels. 

Me would not adopt anything tailored specifically for 5th to 6th grades. 

Information Availability of information about product. 

What is your estimate of the number of people in this school who may have heard 
of it ( Man: A Course of Study )? 

Virtually everyone. 

About half. 

Very few. 

^No one. 

Materials Availability of materials necessary to use product. 

A small, closed-circuit videotape system costs $1800 to Install. Does this school 
have access to one? 

Yes 

No, but we could easily manage to install one or get access to one if we 

found we needed it. 

^No, it would be difficult for us to install one or get access to one, even 

if we needed it. 

^No, and we almost cetainly could not install one or get access to one. 

We definitely could not install one or get access to one. 

Peer Opinion — Degree to which teachers, other teachers, approve of product. 

What do you perceive to be the feeling in other schools with which you have 
contact as regards the teaching or cultural anthropology to 5th and 6th grades? 

Highly valued. 

^Valued, but not highly. 

Not valued. 

Don't know. 

Would their feeling affect whether you would want to teach it? 
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Personnel — Availability of adequate and adequately trained personnel for use 
0^ product. 

Inservlce training frequently requires coordination by some district or 
building level person—usually a curriculum specialist. Would this be a problem 
in this school? 

^No--we could get a coordinator. 



This would be a minor problem, but I expect we could find someone. 



This would be a rnajor problem--we probably couldn't find anyone to do this. 



We would not be able to find a coordinator for inservlce training. 

Priority — Degree of pressure to adopt alternative products with higher 
priorities. 

Right now, are there pressures on you— either direct or indirect--to make 
specific adoptions. (Interviewer: here we're getting at priorities of outsiders 
which are competing with Minicourse and IGE). 

No 



Yes, what? 



(How strong are these pressures?) 



Product/Process — Degree of necessity of adopting a method as opposed to 
adopting Just materials which would fit any or an existing method. 

In this school, would there be any objection to adoptinq materials which require 
that you develop your own program for using them, i.e. as opposed to adopting 
a method along with the materials? 

There would be no objection. 

There would be some objection, but such materials could be adopted anyway. 



There would be much objection— we would probably not make such an adoption. 
There would be much objection— we definitely would not make such an adoption. 
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Programmed — Appropriateness of degree to which product is programmed. 

What is the feeling in this school about the use of programmed learning? 

We always use only those methods and materials which involve programmed 

learning, at least to some extent. 

We u sually use only methods and materials which involve programmed learning, 

at least to some extent. 

We prefer to use methods and materials which involve programmed learning, 

but we frequently make exceptions. 

We prefer to adopt methods and materials which do_ not. involve programmed 

learning. 

Doesn't matter either way. 

Purchase Access i bi 1 i ty — Acceptability of conditions for purchase; ease of 
actually obtaining product; possiblity of cooperative sharing with other 
schools. 

Is it difficult for this school to go through the mechanics of making a major 
purchase from a major publishing company (e.g. MacMillan)? 

^No, once we've decided to make the purchase, we have no problems getting what 

we want froir the publisher. 

^Yes, this involves a certain amount of hassle for someone in the school. 

Yes, this is a major hassle for someone in the school. 

Yes, this is such a problem, we don't ever do it. 

Reorganization Required Degree to which product requires changes in purchasing 
unit. 

In this school, would adoption of a new elementary (5th-6th grade) social studies 
program require reorganization of an already-existing inservice program? 

Uo 

^Probably not, since our present program is very flexible. 

^Yes, it would probably require some minor reorganization. 

^Yes, it would probably require some major reorganization. 

Yes, it would probably require abandoning the present program. 
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Self-Sufflcent -- Extent to which product is complete as is, degree to which pro- 
duct is "teacher proof." 

In this school, is there any resistance to programs 4nd materials which are 
dependent on teacher and/or administrator abilities for their success? 

Ho 

Yes, there is some resistance. 

Yes, there is much resistance to such programs. 

^Yes, such programs are never adopted here. 

Sequential Appropriateness of required pre-requi sites to use of product. 

What is the feeling in this school about whether units of a one-year elementary 
(5th-6th grade) social studies course should be sequential? 

In itself, this is not important one way or the other. 

We would prefer that such a course be sequential. 

Vie prefer a course which is not sequential but we could also use a course 

which is. 

We are unlikely to use any such course if units are sequential. 

We would not adopt a course with sequential units. 

Space Required — Appropriateness of amount and kind of physical space required 
for use of product. 

In the 5th-6th grade classrooms in this school, is there room to store two 
bookcases of ma;.erials for social studies? 

^Yes 

Not really, but we could manage to find room. 

^No, it would be very difficult to find room. 

No, it would be impossible to find room for this. 

Structured -- Appropriateness of degree of flexibility of curriculum. 

What is the attitude in this school toward using programs which are unstructured, 
i.e. no predetermined, set procedures? 
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This approach is acceptable to everyone. 

Some object to this, but this probably would not prevent us from adopting 

such a program. 

An unstructured program probably would not be adopted here. 

An unstructured program would not be adopted here. 

Subject Matter Degree of appropriateness of subject matter. 

What is the attitude at this school regarding the teaching of cultural anthropology? 

riost consider that the teaching of cultural anthropology is one of our highest 

priorities. 

_The teaching of cultural anthropology is not of the highest priority for most 

although it is considered important. 

This is not really a priority— the school is more concerned with many other 

areas at this time. 

No one is concerned at this time with teaching cultural anthropology. 

Target Group -- Appropriateness of product's stated target group to district's 
stated needs. 

What is the feeling in this school as regards the need for improving teaching 

at the 5th and/or 6th grade levels (all abilities)? 

There is agreement that there exists a high priority need to do something 
for one or both of these grade levels. 

^Improvement for one or both of these grades is desired but not of the 

highest priority. 

^Improvement for one or both 'Of these grades is not even a significant priority— 

^the school is more concerned with other matters at this time. 

No one is concerned at this time with improvement at one or both of these 

grade levels. 

Theory -- Compatibility or acceptability of theory on which product is based. 
What is the feeling in this school about individualized instruction? 

General agreement that this is a valid, effective approach. 

Some do not agree with this approach. 

Most do not agree with this approach. 
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Virtually no one agrees with this approach. 



Time Required Appropriateness of time required for use of product. 

Consider that a specific elementary (5th-6th grade) social studies course is 
designed as a one-year course (one lesson per day). Is this the schedule this 
school would want for such a course? 

Yes 



This does not exactly fit our needs, although we could use it. 

No, we probably could not use this. 

No, this is way off base; we would not adopt it. 



Training Required -- Degree of acceptablility of amount of teacher training 
required for effective use of product. 

Consider that an elementary (5th-6th grade) social studies program requires 
30 hours of training for teachers; would this be considered a problem at this 
school? 

No, not at all. 



Yes, but only a minor one. 

Yes, such a requirement would probably prevent us from deciding to use the 
course. 

Yes, we would never adopt a course with those training requirements. 



Unit of Adoption -- Level at which adoption decision must be made, ranges from 
TndTvidual teacher, to district level decision; may be independent of level 
of use of product. 

For this school, who must be involved in the decision to adopt elementary (5th and/or 
6th grade) social studies materials? (Check as many as apply) 

^Superintendent or Assistant Superintendent 

^Principal or Asslsstant Principal 

Teachers 

Other (Specify: ) 
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Would you have this any other way? 
No 

^Yes, but the present system is adequate. 

^Yes, the present system causes serious problems when we need to make such 

an adoption. 

Yes, the present system makes it impossible for us to adopt the product we need. 
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APPENDIX B: PRODUCT DESCRIPTIONS* 

Minlcourses are self-contained, multimedia packages designed to develpp 
specific competencies for elementary school teachers. Based on a process 
called microteaching, each Minicourse focuses on a set of carefully defined 
skills which make up a teaching strategy. The teacher practices the skills 
in short, videotaped sessions with a few pupils. In each of the series of 
seven Minlcourses, the teacher first reads in a handbook about the behaviors 
to be learned; then he views an instructional film in which these skills are 
demonstrated; next a model film tests his ability identify each of the 
skills. The teacher then practices the skills, usually with a small group 
of pupils, in a microteaching session that is videotaped for self-evaluation. 
After evaluating his performance, the teacher videotapes his second effort 
to use the new skills in a microteaching situation. This process is repeated 
in each of the four or five lessons that comprise each Minicourse. 

In Minicourse 1; Effective Questioning , participants learn skills which 
Increase the amount and quality of pupil involvement in class discussions. 
Teachers learn techniques such as pausing, redirection, and prompting. 

The Minicourse represents a significant departure from traditional 
approaches to inservice teacher training. It is self-instructional, self- 
contained, multimedia, and targeted to highly specific skills. The first 
change it demands from most schools or districts is in their budgets, since 
start-up costs are high and few districts have a substantial line item for 
teacher training. The Minicourse also requires sophisticated hardware that 
Is not found in all districts. Scheduling and coord "natlng the microteaching 

♦From Turnbull , Brenda J., Thorn, Lorraine J.., and Hutchins, C.L., Promoting 
Change in Schools. San Francisco: Far West Laboratory for Educational Research 
and Development, 1974. 
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♦.esslons requires the part-time efforts of a designated staff member, and 
usually the schedule Is set up in such a way that released time and substitute 
teachers must be provided. 

Man; A Course of Study (MACOS) focuses on three questions: 

What ib human about human beings? 
How did we get that way? 
How can we be made more so? 

The multimedia student materials have been created from ethnographic film 
studies and field research. Using these sources, classroom teachers and 
students explore the roots of man's social behavior through the study of 
selected animal groups and an intensive examination of the Netsillk Eskimo 
society. The curriculum is designed so that organizing Ideas are Introduced 
early and recur periodically. Social science sklllc are emphasized. Students 
simulate the anthropologist's methods of observation, data collection, hypo- 
thesizing, problem finding, and problem solving. 

This curriculum does not fit into a traditional subject area for schools, 
since It contains material frjm anthropology, natural science, and other 
disciplines. Some of the material on evolution and reproduction proves 
controversial in some communities. Teaching with MACOS can demand an unusual 
amount of flexibility. The initial cost of this multimedia curriculum 
appears high, especially since inservice training is required . However, 
several potential impediments to adoption have been addressed by the publisher: 
the training covers both the unfamiliar content and the new methods, and 
the price can be lowered through deferred payment for the materials and 
various special arrangements for the training. 
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IGE alters the traditional organization of school'^. Instead of one 
autonomous teacher in each classroom, there are teams of staff members 
working with large, nongraded groups of pupils; decision making becomes 
cooperative, with a system of staff committees at different administrative 
levels. Instructional processes change in the direction of individualized, 
diagnostic-prescriptive methods. In adopting IGE, a school commits itself 
to a thoroughgoing program of inservice training in which the staff learns 
to work in the new task structure. 

The multiunit organizational plan replaces traditional, 25-pupil class- 
rooms with larger, nongraded units. Each unit has 100 to 150 children in 
a three- to four-year age span, and instruction is handled by a team of a unit 
leader, two or three staff teachers, a first->ear teacher, and an aide, 
helped by a secretary and an intern. The unit leaders work with the building 
principal as an Instructional Improvement Committee that defines the school's 
goals. Policy development and resource management are handled at the 
district level by a Systemwide Policy Committee, which includes principals 
and some unit leaders and teachers, along with central staff. 
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